Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments.

OBJECTIVE: The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool. DESIGN: Qualitative analyses using semi-structured interviews and analysis of narrative comments. SETTING: Main hospital OR suite at a large academic medical center. PARTICIPANTS: faculty, and residents who work in the OR suite. RESULTS: Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies. CONCLUSIONS: Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.

Rater Model Using Signal Detection Theory for Latent Differential Rater Functioning.

Differential rater functioning (DRF) occurs when raters show evidence of exercising differential severity or leniency when scoring examinees within different subgroups. Previous studies of DRF have examined rater bias using manifest variables (e.g., use of covariates) to determine the subgroups. These manifest variables include gender and the ethnicity of the examinee. For example, a rater may score males more severely. Ideally, each rater's severity should be invariant across subgroups. This study examines DRF in the context of latent subgroups that classify possible sources of DRF based on raters' scoring behavior rather than manifest factors. An extension of the latent class signal detection theory (LC-SDT) model for identifying DRF is proposed and examined using real-world data and simulations. Results from real-world data show that the signal detection approach leads to an effective method to identify latent DRF. Simulations with varying sample sizes and conditions of rater precision were shown to recover parameters at an adequate level, supporting its use to identify latent DRF in large-scale data. These findings suggest that the DRF extension of the LC-SDT can be a useful model to examine characteristics of raters and add information that can aid rater training.

Study habits and academic achievement among medical students: A comparison between male and female subjects.

PURPOSE: Study habits of medical students contribute to better understanding learner achievement and success. This study examines the relationship between study habits, including gender and nonacademic factors that affect learning, to investigate their relationship with performance outcomes. METHODS: This study was conducted in March 2015 at the College of Medicine and Applied Medical Science at Taif University. A survey was administered, measuring study habits such as study time, study partners, source of study, breaks, study interruptions, difficulty concentrating, study activity, and delayed study. Comparisons were examined by high and low grade point average (GPA) and by gender. RESULTS: A total of 257 students completed the questionnaire (59% high GPA and 41% low GPA; 50% males and 50% females). Results indicated significant differences for time of study, study materials, study interruptions, study enjoyment between students of high and low GPAs. Gender differences were found for study time, study methods, study breaks, student activity, and delayed study time. CONCLUSIONS: This study found that study habits differ by gender and also have significant impact on performance outcomes of learners. Given that students' academic success is an important outcome for medical schools, these findings call for implementing these results to enhance curricular changes and promote better learning outcomes.

Mentoring Clinical-Year Medical Students: Factors Contributing to Effective Mentoring.

THEORY: Academic mentoring is an effective method of enhancing undergraduate medical student academic performance, research productivity, career planning, and overall satisfaction. HYPOTHESES: This study investigates the relationship between mentor characteristics and mentee academic performance, with an emphasis on identifying students who need special support. METHODS: A cross-sectional study was conducted among fourth-year medical students at King Abdulaziz University Faculty of Medicine undertaking the clinical skills module (CSM) rotation. Mentors included senior and junior faculty members from the Department of Internal Medicine and the Department of Family Medicine. King Abdulaziz University Faculty of Medicine assigned 1 mentor for every 10 medical students. We organized our mentoring program in the following format: (1) an initial group meeting (mentor with all 10 medical students) and (2) subsequent one-on-one meetings (mentor with each mentee alone). We assessed mentor characteristics, student academic performance and satisfaction, and the rate of mentees referred for special support. RESULTS: A total of 184 students completed the CSM rotation. Among these, 90 students responded to the preprogram survey, with 83% reporting that mentoring was important to them. Group meetings and one-on-one meetings were attended by 60% and 49% of all students, respectively. The most frequent type of support required by the participating students was psychological support (12% of mentees). Participation in the mentoring program had no significant effect on student academic performance. Mentor seniority (P = .024) and motivation (P = .002) were significantly associated with the rate of student referral for special support. CONCLUSIONS: This study demonstrated that academic mentoring can be effective in enhancing student outcomes and promoting special support for students. Moreover, mentor and mentee motivation were found to be essential elements of a successful mentoring program.

Making the case for mastery learning assessments: key issues in validation and justification.

Theoretical and empirical support is increasing for mastery learning, in which learners must demonstrate a minimum level of proficiency before completing a given educational unit. Mastery learning approaches aim for uniform achievement of key objectives by allowing learning time to vary and as such are a course-level analogue to broader competency-based curricular strategies. Sound assessment is the cornerstone of mastery learning systems, yet the nature of assessment validity and justification for mastery learning differs in important ways from standard assessment models. Specific validity issues include (1) the need for careful definition of what is meant by "mastery" in terms of learners' achievement or readiness to proceed, the expected retention of mastery over time, and the completeness of content mastery required in a particular unit; (2) validity threats associated with increased retesting; (3) the need for reliability estimates that account for the specific measurement error at the mastery versus nonmastery cut score; and (4) changes in item- and test-level score variance over retesting, which complicate the analysis of evidence related to reliability, internal structure, and relationships to other variables. The positive and negative consequences for learners, educational systems, and patients resulting from the use of mastery learning assessments must be explored to determine whether a given mastery assessment and pass/fail cut score are valid and justified. In this article, the authors outline key considerations for the validation and justification of mastery learning assessments, with the goal of supporting insightful research and sound practice as the mastery model becomes more widespread.